TECHNICAL REPORT SRE it LUNIVESITY

A method of image segmentation has been developed based on creating links between pixels in successive layers of a "pyramid" of reduced-resolution versions of the image. In the original implementation of this method, the links were based on comparing the values of a single feature, (average) gray level, for each pixel. In this note, the method is extended to links based on multiple features, such as color components or neighborhood properties. c The support of the Defense Advanced Research Projects Agency and the U.S. Army Night Vision Laboratory under Contract DRAG-53-76C-O.38 (DARPA Order 3206) is gratefully acknowledged, as is the help of Sherry Palmer in preparing this report. ' a 'C ' '> " . ' " ; -->".. -... .,:;. ;-.. , . .; .,-". ; ;..,.'..-*..... k o A method of image segmentation described in [13 is .. 1 based on creating links between pixels in successive layers of a "pyramid" of reduced-resolution versions of the image. The method makes use of an "overlapped" pyramid in which 2 each level in initially obtained by averaging 4-by-4 blocks of pixels on the level below, where these blocks overlap by 50% in the x and y directions; this implies that each pixel has four "fathers" on the level above it. To segment the image, we link each pixel (at each level of the pyramid) to that one of its four fathers whose gray level is most similar to its own. We now recompute the value of each pixel (on the levels above the image itself) by averaging only those pixels on the level below that are linked to it. Based on these new values, we change the links as necessary; recompute the values based on these new links; and so on, repeating the process until there is no further change (typically after a few iterations). The resulting final links define subtrees of the pyramid, having pixels as leaves and having roots at the, uppermost level, which we usually take to be the 2-by-2 level. Each subtree thus defines a subset of the pixels (its leaves), so that we have segmented the image into (at most) *j four subsets. The linking process need not be used on average gray h Ai level,; we can use any property that can be computed for th pixels of the image, and extend this property to the highe common gray level (four widely spaced levels were used). The red segmentation does not distinguish bushes from shadowed brick; the green segmentation breaks the bushes up into a grass-like and a shadow-like part; and the blue segmentation, besides not distinguishing bushes from shadow, breaks the sunlit brick into arbitrary subclasses. Figure lc shows results when we use two colors at a time% green and blue in the left column, blue and red in the center column, and red and green in the right column. Here, the segmentations confuse parts of the bush class with the shadow class, but the blue/red segmentation gives a fairly good partitioning of the image into the four types of regions. Figure 2a shows a local "busyness" measure computed for each pixel of the red, green, and blue images. [For the abc 3-by-3 neighborhood d e f, this measure is defined by ghi min(la bI + Ib ci + Id el + le fi + Ig hi + Ihil, .a dl + Id gI + lb el + le hi + Ic fl + If il) where the min is used to decrease the response to step edges, while retaining a high response in isotropic "busy" regions.] Figure 2b shows segmentation results based on busyness alone; . the green result, in particular, gives fair discrimination . .between the bushes and the other regions, but the other ':,it q ". ogions themselves as should be expected are not distin, -. O 9u 'shed. Figure 2c shows results when we use two features, . f, Vlevel (in the given band) and busyness; none of these ' j-° aOtt. .